AUTOMATING A HUMAN FACTORS EVALUATION OF GRAPHICAL USER 
INTERFACES FOR NASA APPLICATIONS: AN UPDATE ON CHIMES * 

Jianping Jiang Elizabeth D. Murphy 

Sidney C. Bailin 

CTA INCORPORATED, Rockville, MD 
Walter F. Truszkowski 

NASA-Goddard Space Flight Center, Greenbelt, MD 


ABSTRACT 

Capturing human factors knowledge about the de- 
sign of graphical user interfaces (GUIs) and apply- 
ing this knowledge on-line are the primary objec- 
tives of the Computer-Human Interaction Models 
(CHIMES) project. The current CHIMES pro- 
totype is designed to check a GUI’s compliance 
with industry-standard guidelines, general human 
factors guidelines, and human factors recommen- 
dations on color usage. Following the evaluation, 
CHIMES presents human factors feedback and ad- 
vice to the GUI designer. The paper describes the 
approach to modeling human factors guidelines; 
the system architecture; a new method developed 
to convert quantitative RGB primaries into quali- 
tative color representations; and the potential for 
integrating CHIMES with user interface manage- 
ment systems (UIMS). Both the conceptual ap- 
proach and its implementation are discussed. This 
paper updates the presentation on CHIMES at the 
first International Symposium on Ground Data 
Systems for Spacecraft Control [1]. 
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color. 

1. INTRODUCTION 

An effective user interface (UI) is key to the suc- 
cess of any complex, automated system, such 
as a ground control center for spacecraft oper- 
ations. Many efforts have been made to assist 
user-interface designers in building usable visual 
displays. User-interface prototyping tools provide 
direct interaction between the designer and the 
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design, as well as reduced programming through 
automatic code generation .[10]- Although build- 
ing a user interface requires less effort than ever 
before, many poor designs can still be found be- 
cause they violate human factors design principles 
or industry-standard guidelines, such as the spe- 
cific requirements specified in the OSF/Motif Style 
Guide[§\. Improving poor UI designs requires hu- 
man factors expertise and style-guidelines knowl- 
edge, which are, however, expensive and not 
readily available during design and development. 
Automated human-factors-guidelines and style- 
guidelines compliance checking are promising ap- 
proaches to the problem [8]. 

Under the direction of NASA-Goddard Space 
Flight Center, a series of research and prototyping 
cycles has produced the current fourth-generation 
CHIMES methodology and toolset[7]. A previous 
paper describes the original CHIMES methodol- 
ogy and early prototypes[l]. The current GUI- 
evaluation prototype includes a style-compliance- 
checking tool, i.e. an automated critic that checks 
a design against style rules implemented in the 
tool’s knowledge base. This prototype includes ca- 
pabilities to evaluate conformance to human fac- 
tors guidelines and recommendations on color us- 
age. The tool provides feedback within the con- 
text of identified instances of non-compliance and 
advises the GUI designer on how to modify the 
design. 

In the sections that follow, we describe the im- 
plementation approach used in the current pro- 
totype, the architecture, the implementation of 
compliance-checking rules, and plans for future 
research. Color representation in the knowledge 
base is also described. 



2. IMPLEMENTATION APPROACH 

Human factors guidelines cover many aspects of 
a GUI design, from color selection to screen lay- 
out. Style guidelines provide more specific require- 
ments. For instance, the OSF/Motif Style Guide 
requires a menubar to be placed at the top left of 
a screen. Because so many factors are involved, a 
GUI design’s non-compliance with human factors 
guidelines or style guidelines can take many forms. 
In most cases, the problems of non-compliance 
are independent. Sometimes, however, they are 
inter-connected. For example, using smaller fonts 
may cause poorer legibility of colored symbols. 
Thus, checking non-compliance problems in GUI 
designs demands consideration of many factors 
and diverse responses from a compliance-checking 
program. This requires an effective implementa- 
tion approach to support the complex control of 
switching between blocks of code. 

The conventional programming approach, used by 
systems written in C or FORTRAN, fails to pro- 
vide adequate support, because the code in such a 
system is executed following a predetermined se- 
quence, and a programmer is required to develop 
the control structure, which is very complex in the 
case of non-compliance checking. A more appro- 
priate approach is a rule-based system using for- 
ward chaining, because its control mechanism au- 
tomatically executes the right blocks of code based 
on input data. 

A rule-based system has three components: a data 
memory, a production memory, and an inference 
engine[3]. Data memory stores the input data and 
the current state of knowledge during the prob- 
lem solving process. Production memory stores 
the rules which constitute the program. The in- 
ference engine executes appropriate rules based on 
the configuration of data memory. A rule has two 
parts, a conditional part and an action part. The 
conditional part describes the data memory con- 
figuration for which the rule is appropriate, and 
the action part gives the instruction for changing 
the data memory. During code execution, rules 
that are satisfied by the current content of data 
memory are collected; one of the collected rules 
is then selected based on some selection strategy; 
finally, the selected rule is executed. 

When forward chaining is used in rule-based sys- 
tems, rules are executed whose conditional parts 
are satisfied by the current content of data mem- 
ory. The rules executed are determined by in- 


put data because the content of data memory is a 
transformatin of input data. This frees program- 
mers from writing complex control code, thus pro- 
viding better support for system development. 

By using forward chaining, a compliance checker 
can represent GUI designs as facts in the data 
memory and can model compliance-checking ex- 
pertise in the form of rules which match non- 
compliant patterns. When a GUI design violates 
the guidelines, the facts in the data memory will 
reflect the violation, which will match a condition 
of the compliance-checking rules. This leads the 
inference engine to execute the appropriate pro- 
gram code. 

3. CHIMES ARCHITECTURE 

The current CHIMES prototype architecture, il- 
lustrated in Figure 1, uses a rule-based-system- 
with-forward-chaining approach to checking a 
GUI’s compliance with human factors guidelines 
and style guidelines. There are five modules: de- 
sign acquisition module, compliance-check con- 
troller, knowledge base, advice generator, and user 
interface. 

The purpose of the design acquisition module is 
to acquire GUI design information. It transforms 
a design into input data for CHIMES to evaluate. 
This module consists of two submodules, the auto- 
matic design capture submodule and the manual 
design capture submodule. The automatic design 
capture submodule acquires GUI design informa- 
tion by reading a design description file, such as 
a TAE 1 resource file. The manual design capture 
submodule supplements the automatic design cap- 
ture submodule by capturing additional design in- 
formation, through queries to the GUI designer. 

The primary function of the compliance- check con- 
troller is to initiate a CHIMES evaluation of an 
acquired design. It takes the design informa- 
tion from the design acquisition module, initializes 
the knowledge base, converts the information into 
facts, asserts the facts into the knowledge base, 
activates compliance checking rules, and initiates 
the evaluation. 

The knowledge base is a key component of the 
CHIMES prototype system. It contains the rules 

*TAE is a trademark of the National Aeronautics and 
Space Administration. The Transportable Applications 
Environment (TAE) Plus is a user interface development 
and management environment [10]. 
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Figure 1: CHIMES Architecture 




















that model compliance-checking expertise and the 
facts that represent the user-interface design be- 
ing evaluated. Upon receiving the evaluation in- 
struction from the compliance-check controller, 
the knowledge base performs the compliance check 
and calls the advice generator to generate appli- 
cable advice. 

The advice generator presents the results of com- 
pliance checking to the GUI designer. It prepares 
advice based on the results of compliance check- 
ing. The advice is then displayed in the CHIMES 
user interface, with an indication of the display 
element that triggered the advice. This provides 
advice in the context of the problem. 

The user interface allows the GUI designer to initi- 
ate compliance checking. It also serves to present 
the compliance checking results to the GUI de- 
signer. For ease of use, all user interactions in the 
prototype system occur via mouse positioning and 
clicking. 

The knowledge base is implemented by using 
CLIPS 2 [4], which supports development of a rule- 
based system with forward chaining. The user in- 
terface is developed by using TAE Plus with X 
Window System 3 and OSF/Motif 4 . The design ac- 
quisition module, the compliance check controller 
and the advice generator are written in C and 
C++. 

Interaction of Modules. Compliance checking be- 
gins with the design-acquisition module, which 
currently acquires a user- interface representation 
from a TAE resource file and transforms it into 
input data for the CHIMES evaluation. Once the 
GUI representation has been captured, it is sent 
to the compliance-check controller, which issues 
requests for compliance checks to the CHIMES 
knowledge base. If violations of the guidelines are 
found, advice is generated on how to improve the 
design. 

4. IMPLEMENTATION OF RULES 

Checking a GUI design’s compliance with human 
factors guidelines and style guidelines is performed 

2 CLIPS (C Language Integrated Production System) 
is a trademark of the National Aeronautics and Space 
Administration. 

3 X Window System is a trademark of the Massachusetts 
Institute of Technology. 

4 OSF/Motif is a trademark of the Open Software 
Foundation. 


in the knowledge base, where a design is rep- 
resented as facts in the data memory, and the 
compliance-check expertise as rules in the pro- 
duction memory. This section discusses the im- 
plementation of compliance-check expertise in the 
knowledge base. 

CHIMES’ compliance-checking rules are devel- 
oped by modeling non-compliance patterns. These 
patterns take various forms, such as geographic vi- 
olations; for instance, placing a menubar at the 
bottom of the screen violates the OSF /Motif style 
guidelines. Overuse of design elements, for in- 
stance, using five fonts in a screen or color mis- 
matchings, for instance, choosing black for the 
background color and red for the foreground color, 
can yield poor legibility or visual discomfort[ll]. 

The non-compliance patterns are described in the 
conditional parts of the rules, which catch the vi- 
olations. For example, a rule examining menubar 
position has the following form: 

(defrule check-menubar-location 
( goal-is check-menubar-location) 

(item (name ?name) (type menubar)) 

(loc (name ?name ) (x ?x&:( 0 ?x)) 

(y ?y&:& 0 ?y))) 

=> 

(advice “According to OSF/Motif Style 
Guide, the menubar should be placed at the 
top left of the window. ”)) 

The conditional part of this rule defines three con- 
ditions for the rule to be executed: 1) the knowl- 
edge base is checking menubar location; 2) there 
is a menubar item in the data memory; and 3) the 
menubar is not located at the upper left corner 
of the screen. The action part of the rule specifies 
which piece of advice should be generated if a non- 
compliant menubar is found. This rule catches any 
violation of standard menubar placement, which 
may have the following form in the data memory: 

(item (name mb) (type menubar )) 

( loc (name mb) (x 170) (y 10)) 

While many of the violation patterns can be repre- 
sented in the knowledge base directly, color is an 
exception, because the form of color representa- 
tion used for electronic display is not appropriate 
for evaluation by rule-based systems. 

Color is produced on an electronic display by ad- 
ditive mixture of three primary colors, red, green 
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and blue. Each primary color is associated with 
a number to indicate its intensity. For instance, 
yellow is represented in OSF/Motif as (255, 255, 
0), indicating that yellow is the mixture of red and 
green. 

A prominent problem that prevents use of this 
representation in rule-based systems is the size of 
the color space, which makes developing rules a 
formidable task. For example, a color space with 
each primary color ranging from 0 to 255 consists 
of more than sixteen million colors. It would re- 
quire more than sixty trillion rules to cover all the 
possible color combinations. 

To reduce the size of color space, a linguistic color 
model, the color-naming system (CNS)[2], is used 
in CHIMES. The CNS model consists of about 
three hundred color names. Each name contains 
information on hue, saturation, and lightness. An 
example of a CNS name is “ very light vivid yel- 
low” where the saturation modifier vivid, and the 
lightness modifier, very light , supply more infor- 
mation on the hue yellow . 

To use CNS color names in the knowledge base, 
color representation conversion is needed, because 
most computer systems use red-green-blue pri- 
mary colors to represent color. The conversion 
involves three color models, the red-green-blue 
(RGB) model [5], the hue-saturation- value (HSV) 
model [11] and the CNS model. First, a color 
is translated into the representation of the RGB 
model, which uses red-green-blue primary colors 
but with the intensity range between 0 to 1. Sec- 
ond, a color’s RGB representation is translated 
into the HSV representation using a transforma- 
tion formuIa[12]. By using the HSV representa- 
tion, a color’s hue is specified in terms of degree 
on a color ring; saturation and value, or “light- 
ness” which indicates color intensity, are described 
within the range from 0 to 1. At last, the HSV rep- 
resentation is mapped into a CNS color name by 
using a mapping matrix. For example, A yellow, 
represented as (255, 255, 0) in OSF/Motif, can be 
translated into the RGB representation (1, 1, 0). 
Then, the RGB representation can be translated 
into an HSV representation (60, 1, 1), which sub- 
sequently can be mapped into a CNS name, very 
light vivid yellow , because the 60th degree hue is 
mapped to yellow on the color ring, and the high 
lightness and saturation map to the modifiers, very 
light and vivid. 

When checking a GUI design’s compliance with 


human factors guidelines on general color usage, 
color representation conversion is done by the 
compliance-check controller. It translates all the 
colors in the design into CNS representation, and 
stores them in the data memory, where they may 
trigger compliance-checking rules. 

5. FUTURE RESEARCH PLANS 

Automated compliance checking is a promising ap- 
proach to developing a utility in UIMS, such as 
NASA’s TAE Plus. Such a utility can help a 
GUI designer build more usable user interfaces. It 
can extract information from the system resources; 
evaluate the design against guidelines; and pro- 
vide advice to the GUI designer on how to im- 
prove the design. The GUI designer can then use 
user-interface building capabilities in the UIMS to 
correct design flaws and produce a better user in- 
terface design. The current CHIMES prototype 
is our first step towards developing such a utility. 
Future plans include expanding the compliance- 
checking capability, developing a generic input in- 
terface, and creating a style guidelines description 
language. 

The scope of a compliance-checking capability is 
key to its success. Currently, the CHIMES com- 
pliance check is limited to one screen and to the 
“look,” i.e. the visual aspect, of a GUI design. 
We are going to expand the CHIMES compliance- 
checking capability so that it can evaluate a de- 
sign with multiple screens. To lay the foundation 
for achieving this objective, we plan to develop 
heuristics on consistency of layout, labeling, and 
color usage. We also plan to develop heuristics on 
evaluating the “feel,” i.e. the behavioral aspect, 
of a GUI design, which requires recording a GUI’s 
behavior during execution time. 

We plan to expand the CHIMES design-input ca- 
pability. Currently, CHIMES extracts design data 
from a TAE resource file. An interface to a stan- 
dard user interface description language, such as 
UIL[6], will make CHIMES capable of evaluating 
GUI designs developed in other UIMS environ- 
ments. 

A style guidelines description language is needed 
when developing a utility to evaluate GUI designs 
against various style guidelines. The idea of evalu- 
ating GUIs against various guidelines is supported 
by an existing CHIMES feature, that the designer 
can change the CHIMES compliance-checking be- 
havior by changing its rules in the knowledge base 
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without needing to re-compile. With an ade- 
quate guidelines description language and a trans- 
lator that translates guidelines into compliance- 
checking rules, a compliance-checking utility will 
be able to check a design’s compliance with style 
guidelines such as those developed for OSF/Motif, 
Open Look, or other customized toolkits. 
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